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Abstract p 

Reliability and its interdependence with testing are important topics for development 
and manufacturing of successful products. This generally accepted fact is not only a 
technical statement, but must be also seen in the light of "Human Factors." While the 
background for this paper is the experience gained with electromechanical/electronic 
space products, including control and system considerations, it is believed that the 
content could be also of interest for other fields. 

Introduction 

The consideration of the interdependence of reliability and testing is an important 
subject for the success of development and production. The background for this paper 
is the experience gained with electromechanical/electronic space products, including 
control and system investigation. Besides "Technical Necessities," the "Human 
Factors" have to be treated. Some examples of the positive or negative influence are 
given. 

Reliability 

Reliability can be described as the property of a product to fulfill the intended task and 
duty in a given environment for a defined duration with a certain probability of success. 
Reliability has to be therefore an inherent quality of a product which can be - and for 
space products, has to be - proven by appropriate testing. The required reliability has 
to be a major design driver from the very beginning, the first ideas and feasibility 
studies. This work has to also include investigations in to how the quality, reliability, 
and the performance of the intended product can be determined and measured. 

Mostly, the already available equipment for measurements, testing and production as 
well as design aids and experience, are points of main consideration. The same is 
true for proven processes for design and production. 

Reliability calculations have to accompany design and development. The results of 
those calculations may lead to a preferred design. But caution is necessary whether 
the reliability numbers assumed can be realized with the available technology, parts 
and other resources. Already existing lists of acceptable materials, parts, subunits, 
software programs and so on, and of qualified vendors should be obeyed. For those 
elements, there are also proven reliability and life numbers available. A formal quality 
assurance approach with written procedures and instructions is most helpful for 
achieving reliability. All relevant methods and steps described there should be 
implemented regularly in the course of development and production phases. 
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A good definition of the work to be done with specifications demanding the necessary 
things only (with reasonable margins) are important for quality and schedule. The 
most experienced people should establish the concerned documents. Quick changes 
introduced after discovery of a disadvantage should be avoided. Reliability needs to 
scrutinize all aspects of a change. This is important throughout all phases of a 
program. Hardware and software must be treated equally from the standpoint of 
quality and reliability. 

Cost and schedule restrictions may limit the efforts towards reliability. Therefore, 
proven principles or even existing design approaches should be used as much as 
possible in an actual program. Basic research and alternate design investigations 
should be studied independent of flight programs with less pressure on schedule but 
with clear aims. 

It is also important that the group, working on a certain product, has the necessary 
knowledge of the system in which this product is used. Generally, this reduces 
interface problems and aids the definition and improvement of meaningful 
specifications. It is also speeding up a development process and is a source of 
motivation. In this context, interface questions must be thoroughly considered and 
defined. The validity and strictness of interfaces must be continuously discussed and 
checked by testing in early phases of a development. This is especially important in 
connection with subcontractors that work outside of the company or even abroad. 
Formal controls are inevitable. A responsible person has to overlook in a competent 
way both sides of an interface. 

Important subcontractors have to be incorporated in quality and reliability efforts from 
the very beginning. In many cases, education measures have to be taken. If 
subcontractors have their own quality assurance system, this has to be checked in the 
sense of compatibility. 

If there are doubts in the quality of sections of a product, technological samples, 
thoroughly investigated and tested, are a help to improve certain capabilities early in a 
program or to avoid the initially planned technology in the further development. 

Often "small things" are overlooked: a wire which is not secured, a too high or too low 
torque applied for fastening a screw. The most ingenius design is endangered by 
these inconsequential items. 

Providing redundancies is not always a help in achieving reliability. Switch-over units 
from the primary to the redundant system and back, may be of the single point failure 
category. In hot redundancies - primary and redundant systems work in parallel - the 
testing needs special attention and a high degree of knowledge. 

Besides the philosophy of redundancies, the considerations of alternative and/or 
down-graded modes of operation can be fruitful. With the advent of high telemetry 
transmission rates and powerful ground-based computers, for instance, control loops 
may be even closed via the ground control (of course taking into calculation the "dead 
time" of transmissions - already in the development phase). 
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In many cases, reliability and life of a product have a certain dependence to each 
other. This statement is valid for products that exhibit a random failure behavior. In 
contrast, other products have a certain life expectancy due to wear-out, to fatigue 
processes, to evaporation, and so on. Some products show, in addition, a non-failure 
period, a time span, in which no failure should occur, a property that could be 
important to improve reliability decisively. 

An important advice is: start with a concept as simple as possible. When arriving at 
detailed problems, every approach will demand the full capability of the personnel. 
Normally, a simple approach is also an elegant one that is good for the pride of all 
involved. Here are some examples for electromechanical/electronic devices for the 
principles just mentioned: 

Elimination of caging devices where conceivable (avoiding of single point 
failures, often at the very beginning of a mission). Instead, damping of main 
resonances, limitation of the excursions of oscillations, and similar methods 
should be developed. 

Suspensions and passive actuation can be often combined; a bearing may also 
act as a spring and vice versa. 

Preference of drive systems that require no additional bearing suspension. If 
redundant driving is required, two motors/windings on one axis should be 
applied. (No mechanical switch-over.) 

Selection of drive principles that promise straightforward control, e.g., position 
drives instead of torque drives if better suited. 

Definition of favorable mechanical/electronic interfaces, in view of separate 
testing capabilities of mechanics and electronics. 

Preparation of modular concepts to facilitate development, production, quality 
control and testing. The utilization of proven modules or derivatives of such 
modules enhance credibility of an approach. 

Selection of materials based not only on mass and strength but also on 
machinability, handling precautions, thermal characteristics, surface treatment 
properties etc. 

Utilization of lubrication schemes that work essentially independent of 
acceleration levels and directions as well as under air pressure or vacuum. 

Optimization of reliability by avoiding single point failure possibilities. 
Redundancy switch-over should be passive, if conceivable. 

Valid testability is a must in consideration of any approach for designing a 
product. 

Of course, this list can be carried on, depending on the product to be discussed. 
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in this context, it should be borne in mind that there is a natural pride of "own designs," 
"own ways" which is essential for motivation. But especially here, a constructive 
criticism should be not excluded. Fair comparisons with other designs, different 
approaches, and alternative ways are very helpful. 

Also the effect of a "patent philosophy" can hinder the search for the optimal solution. 
This is true for both aspects: other companies have patents or applied for patents that 
complicate the development, especially if this situation is not known at the beginning 
of a work, or a co-worker in your company has made an invention that he tries to 
realize (also for reasons of financial benefits), even if it is not the best solution. 

Friendliness to each other must not lead to hiding of problem areas and keeping 
secret of incidents. Those should be honored who have the courage to say 
unpleasant truths - in time and in a friendly way. 

A climate of confidence, but also of sound criticism and reasonable competition 
between ministries, agencies, companies, institutes and so forth is important. Political 
considerations should be excluded as much as possible. Technical and scientific 
problems have to be tackled on the grounds of natural laws and not by investigation of 
the opinion of higher ranking people, inside or outside the company. 

Dedicated, educated, well trained, experienced, motivated and co-operative people 
are of utmost importance through all levels of a hierarchy in the course of the design, 
development, production and testing phases. Clear responsibilities, added by strict 
documentation and control are most important prerequisites for reliability. 

Testing 

Quality and reliability are no substitutes for testing. Testing is no substitute for quality 
and reliability. Both have to go hand in hand to reach the goal providing a quality 
product according to specifications. 

Meaningful testing has to accompany a program in all phases: feasibility studies, 
design, development, qualification and manufacturing. Tests must be carefully 
planned and performed, the test results documented. 

Testing represents also a significant part of the costs of a program, but the proof of 
reliability - or sometimes the contrary - is essential for the success of a mission. 

In many cases, designing of a product seems to be a clear sequence of efforts but 
valid testing can be very complex, it is no problem to invent irrelevant tests, to 
misinterpret results, to fail to recognize the significance of an event, and to overtest or 
to undertest. Inadequate testing may even obscure weak points in a design instead of 
bringing them to light. Too long and too severe testing may reduce reliability and life 
of a product, which is of course not intended. 

The purpose of testing, realized in a test program, must be to prove the product's 
specified performance under a predicted environment. In addition, the reliability over 
the required life time has to be demonstrated (mostly in combination with justified 
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calculations). No aspect must be overlooked. Therefore, a test program has to be 
considered as an equally important part of a development, a qualification and a 
production process. 

To achieve the intended purpose of a test program, a certain formality is 
indispensable. This includes written detailed instructions, the co-operation of 
independent and experienced quality assurance and quality control people, and 
written fail/success criteria. The strictness, formality and severity of the tests have to 
increase with the phases of a program from feasibility tests to qualification, 
acceptance, and life tests. 

It is essential to employ experienced people who are able to bear the involved 
responsibility and to set the levels and the sequence of testing. Wrong judgment 
concerning test specification can result in both, damage of the product and schedule 
problems (besides cost overshoots). Adequate test facilities and experienced 
engineers are prerequisites for valid testing, 

The above considerations are valid for the following categories of tests; 

Development Tests (probably also Feasibility Tests) 

Qualification Tests 
Acceptance Tests 

Refurbishment Tests and Differential Tests 
Life Tests, both Real Time or Accelerated 
Development Tests 

These tests are performed usually by design and development engineers to prove the 
validity of design approaches and the technologies involved. Parts and subassembly 
tests could be performed if deemed necessary. Therefore, these tests are not formal, 
but very useful to develop test procedures for later formal tests, to make quality control 
and quality assurance people acquainted with the actual development, and to test - in 
a later development phase - with more mature models, the ability to meet qualification 
specifications. In certain cases, such a pre-qualification tested model is utilized for a 
life test to gain as early as possible additional information on the long-term behavior. 
Of course, adequate written reporting is necessary as reference for later formal testing. 

With the growing complexity of satellites, the Electro-Magnetic-Compatibility (EMC) 
has become of utmost importance. Therefore, strict testing and well-understood 
improvements to the necessary level (with suitable margins) are required in the course 
of the development phase. Deficiencies that would be detected in later phases only 
could force basic changes in a design with a result of time delays and cost problems. 

Qu alificat i o n T est 

This test is crucial to the success of a project. The objectives of a qualification shall be 
to prove that a model built according to the established manufacturing documentation 
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meets all specifications and requirements in the full range of the expected 
environment - with reasonable margins. The results of functional tests before, after, or 
- if appropriate - during, the exposure to environmental conditions must be within 
prescribed limits. The sequence of tests should be similar to that in an actual mission: 
first vibration, acoustic noise and shock tests, then thermal (vacuum) tests. Often 
mechanisms and the associated electronics have to meet different specifications, 
which must be obeyed in test set-ups. 

Qualification by similarity with an already qualified device is often regarded as being 
acceptable. If the main argument is time and money saving, one should be especially 
cautious. In mechanical units, even minor changes may cause catastrophic results. 
Even another cleaning process, another preload of bearings, or "identical parts" 
supplied by another vendor must be carefully considered to avoid unexpected failures. 
Therefore, the qualification by similarity of mechanisms should be the exception and 
only adopted in fully justified cases. 

For electronic units, qualification by similarity is a more valid approach. For instance, 
the fixation of printed circuit boards or the use of special housing configurations are 
examples for acceptable cases. 

A passed qualification test becomes the coronation of a successful development and 
the final demonstration of its reliability, quality and integrity. Such a test must be 
carried out with rigorous formality and obstinate strictness under the control of co- 
workers of the product assurance system. The participation of representatives of the 
customer at qualification tests underlines further the significance. 

Depending on the general philosophy or the special situation, a qualification model 
may be either stripped and examined thoroughly for whether the tests have done any 
harm to the qualification model, or used as a formal life test model, or refurbished to a 
flight model (for instance by exchanging bearings with subsequent acceptance 
testing). 

Of course, a formal qualification report must be established. 

Acceptance Test 

While the qualification test demonstrates the quality, reliability and suitability of a 
design, acceptance testing should prove the quality and integrity of an individual flight 
model, built to the production specifications and procedures. 

Main objectives of this test are therefore: 

Demonstration in an integral way that the tested model is free of workmanship 
errors. 

Proof that the tested model complies with the manufacturing and quality control 
processes, the functional performance requirements, the dimensional, weight and 
interface specifications. 
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This test should be performed under the maximum predicted operational levels (in 
some cases with small margins) concerning environment. Like the qualification test, 
acceptance testing must be carried out in a strict formal way according to written 
instructions under the responsibility of quality assurance people and often under 
supervision of representatives of the customer. A formal acceptance test report must 
be provided in which all important observations and deviations have to be noted. This 
report has to serve during a mission as the main reference if anomalies should occur. 

Refurbishment Tests and Differential Tests 

Refurbishment tests are essentially formal acceptance tests that are carried out on 
models, built to flight standard after their qualification testing, repair, replacement of 
parts and long storage, to prove the flight worthiness. 

Differential tests are performed on already acceptance tested flight models if it has - for 
instance - become necessary to introduce minor changes into performance 
specifications, to marginally change the mechanical interface and so on. 

After careful consideration, a differential test comprises a certain part of an acceptance 
test only. The results of those tests have to be documented as an appendix to the 
acceptance test report of the unit already formally acceptance tested. 

Life Tests. Real Time or Accelerated 

Life tests on themselves are no proof of reliability, quality and life. They make sense 
only, if before, by life calculations, adequate applying of proven technologies, taking 
into account experience gained with similar equipment, and other considerations, a 
basis for such a life test has been established. This becomes clearer if one is aware of 
the fact that a life test with one or some life test models will give no insight in the 
statistical behavior: If one would try to prove that a design has a life of 10 years with 
99% probability, he would need in the order of 1000 life test models to get an answer 
with 1 0% accuracy. Nobody has the money and the time to work this way. And if such 
a test would fail, the money and the time would be lost with the result only that the 
developed design does not meet the life requirement. 

Therefore, a life test can be a proof only for the fact that there is no basic life limiting 
effect present in a design. (Of course the psychological aspects of positive results of a 
life test should be not underestimated.) 

Life testing, both real time or accelerated, needs careful consideration and planning. 

A certain formality and control has to take place. Life test models have to be 
representative concerning the life determining and life limiting features. This needs 
careful judgment. 

While real-time life test planning is mostly straightforward, the establishing of 
accelerated life test procedures needs a profound knowledge of the test item and the 
mission conditions to define acceleration factors, modes of operation, and so on. For 
instance, a truly representative accelerated life test with a liquid lubrication system is 
not possible, because acceleration factors for temperature and speed cannot be 
defined. 
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Typical application for accelerated life testing are units with dry lubrication that are 
activated non-continuously. Here the silent periods between actuations can be 
shortened nearly to zero. For instance, the time interval between the steps including 
the transient time of a solar array drive can be eliminated. Here acceleration factors of 
up to two orders of magnitude could result and could be used with a convincing 
justification of the validity. 

Single Tests to be Performed within the above Test Categories 

The following single tests could be part of the above named test categories. (It should 

be mentioned at the very beginning that not necessarily all these single tests have to 

be performed in any case. Thorough considerations should aid reasonable 

decisions.) 

Functional and Performance Tests 
Mechanical Environmental Tests 

Static Load and Constant Acceleration 
Acoustic 

Sinusoidal Vibration 
Random Vibration 
Shock 

Thermal respectively Thermal Vacuum Tests 
Storage Tests 

EMC (Electro-Magnetic Compatibility) Tests 
Functional and Performance Tests 

These tests comprise measurements of mass, dimensions, electrical properties, 
redundancy switch-overs, safety provisions, different operational modes and so on. 
The results should meet the specifications and requirements. They are a reference for 
the comparison of properties before and after exposure to environmental conditions. 

Mechanical Environmental Test 

Static load and constant acceleration tests are necessary when mechanisms should 
bear loads, as instruments and solar generators, even if unloading (caging) devices 
are used. If vibration tests are harder than static load tests, the latter could be deleted. 

Acoustic tests are especially recommended when the test items are mounted at the 
outside of a satellite's skin and large surfaces - acting as "microphones" - are involved. 

Sinusoidal vibrations with low levels are an important development tool for detecting 
resonances and their damping, investigation of interactions, and so on. Here, 
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overtesting could easily occur, therefore caution is necessary. In qualification testing, 
a frequency sweep could be important as proof for integrity. 

Random vibration tests are decisive for proving the quality and integrity of a unit. 
Spectral density functions and levels must be carefully chosen to avoid unnecessary 
overtesting. 

Shock testing should simulate the insensitivity against shocks emanating, for instance, 
from pyrotechnic devices. On a case to case basis, it must be decided whether 
vibration tests are more severe and, as a consequence, these tests could be deleted. 

Thermal and thermal vacuum tests are the most important ones concerning 
performance during a mission. Thermal tests in a normal atmosphere are often 
sufficient for equipment with a hermetic housing. But at least for qualification, thermal 
vacuum testing should be mandatory. Especially for simulating eclipse situations, the 
introduction of thermal gradients could be important for searching weak points in a 
design as well as detecting errors in manufacturing. Often tests with rapid gradients of 
temperature changes - while the equipment is in operation - are touchstones for 
quality, reliability and integrity. Non-operation tests at temperatures exceeding the 
operational range could be necessary to demonstrate functional survival. 


Storage tests must be subdivided into short term and long term storage tests. Short 
term storage tests should generally simulate the sequence of satellite launch and unit 
cold start. This is done best by storing the equipment after subjecting it to vibration 
testing at a defined low temperature for a certain time interval with subsequent cold 
start. Normally these tests could be the first step in a thermal or thermal vacuum test. 

Long-term storage tests could be necessary for equipment that could be dormant for 
long periods of a mission. For instance, a redundant unit checked out in orbit should 
work properly on switch-on also near the end of a mission. The definition of such a 
test needs much experience and a profound knowledge of the equipment. Careful 
consideration of acceleration possibilities is necessary to finish this test in a time span 
fitting with the program. Good candidates for a storage test are qualification models if 
not needed for life testing. The acceleration methods and factors depend on the 
probable failure mode. For instance, if cold welding in a bearing would be the 
suspicion, a high storage temperature under vacuum with perhaps some micro 
vibrations could be employed. 

EMC-tests become more and more important with the growing complexity of satellites. 
Well-educated and experienced people with adequate test equipment must be 
provided to perform valid testing. Such specialists should also influence the design 
and the packaging of electronic equipment, the shielding of motors and so on from the 
very beginning to save cost and time. 

Test Program and Test Sequence 

The Table for Tests explains best which tests are recommended and which are 
mandatory. The tests are also indicated in the sequence to be followed in each of the 
categories. 
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It should be stressed here again that formal testing under control of quality assurance 
and according to written procedures is obligatory for qualification, acceptance and 
refurbishment testing. This cannot be overemphasized. Test reports must be 
complete showing incidents and how they were dealt with, not only the "nice portions" 
of a test. Clear success and failure criteria must be established and followed. 

I fi sLEacilttifis 

The careful preparation of testing with continued improving measures and the 
performance of the tests strictly according to the plans and procedures must find its 
counterpart in excellent testing facilities, continuously and strictly maintained and 
controlled by quality assurance according to written standards and procedures. 
Conscientious, dedicated and experienced engineers must run such facilities with 
indefatigable discipline. The importance of test facilities which are beyond each doubt 
of their top condition cannot be overestimated. The control by quality assurance 
people should be not regarded as an unavoidable burden, but as a help of a totally 
independent but most interested authority to achieve and guarantee excellent quality. 
Out-of-company test facilities have to be monitored, too by experts of the own company 
and often also by those of customers. Quality products need quality testing facilities! 

Conclusion 

It was not possible to treat all aspects of this complex subject in a short paper. Of 
course, the pronunciation of the different facets is characterized by the personal 
experience both enjoyed and suffered. 

It is hoped that the interrelation between Reliability and Testing has become clearer by 
presenting this paper. Reliability cannot be tested into a product but is an inherent 
quality characteristic. Formal testing is the final significant proof of quality and 
reliability. 

Dedicated, educated, well-trained, experienced, responsible and motivated people 
are the main key to quality and to reliability. 
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Table for Tests 


Tests and their Sequence 

Test Category 


Dev. 

dual. 

Accept. 

Refurb. 

Life 


Test 

Test 

Test 

Test 

Test 

Functional Test 

M 

M 

M 

M 


Performance Test 

R 

M 

M 

M 

M 

EMC-Test 

M 

M 




Static Load Test 

R 

M 

M 

M 

M 

Acoustic Test 

R 

M 




Sinusoidal Vibration Test 

M 

M 




Random Vibration Test 

M 

M 

M 

M 

M 

Shock Test 

R 

M 




Performance Test 

R 

M 

M 

M 

M 

Storage Short Term Test 

R 

M 

M 

M 

M 

Thermal Vac. Perf. Test 

R 

M 

M 

M 

M 

Duration Test 





M 

Performance Test 





M 

Strip 

R 




R 


Legend: M: Mandatory, if applicable; R: Recommended 
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